Guided Reading in L. B. Primary School Alison Browne, Robin Halbert, Georgie Perigny, Cavelle Porter, Wanda Quigg
Evaluation Question: Is Guided Reading effective? Sub questions: 1. Is there fidelity of implementation for the Guided Reading program according to the guidelines established by Fountas and Pinnell (1996) 2. Are students who participate in the program demonstrating improvements in reading skills over the school year?
Qualitative Measures Semi-Structured Interview The interview will be used to measure program fidelity. It will address the constructs of organization/ planning, training, and overall impact. Administrators and teachers will be interviewed by the researchers on a one to one basis. In conducting the semi-structured interviews, we will use a pre-established set of questions to guide the process, yet be open to the order in which questions are asked. An advantage to semi-structured interviews is that they allow interviewers to ask probing questions and to proceed in a manner that has a natural, conversational flow (Leedy & Ormrod, 2010). A disadvantage of the interview process is that researchers are asking participants about events that are in the past, requiring that interviewees rely on their memories, which are often subject to a considerable amount of distortion (Leedy & Ormrod, 2010). Interviews can also take more time than alternate measures, such as the focus group, however as we are only conducting four interviews, we felt that it would provide more insight into each individual’s lived experience than would a focus group situation, in which the limitations of focus groups noted below might come into play.
Demographics of group Teachers - The three teachers who have Guided Reading programs at the school will be interviewed. These teachers are between the ages of 35 and 45. One teacher has been teaching for 5 years, one for 10 years, and the third for 22 years. All three of the teachers are female and all three have been teaching primary school throughout their careers. The teacher with 22 years experience is currently enrolled in a distance Master's Degree Program, specializing in educational technology. The other two teachers have bachelor's degrees in education, specializing in elementary education. All three teachers have spent their entire careers at L. B. Primary School.
Principal/Administrator - The principal of L. B. Primary School is male, aged 63, and has been with the School Board for 40 years. He recently transferred to L. B. Primary School, having spent all of his career in high schools, first as a high school chemistry teacher, then as a high school principal. He has a Master's degree in Policy and Administration.
There will be 8 questions to measure the constructs.
Interview Questions Organization and Planning
What kind of reading materials are in place and how are they being utilized?
What time and space has been set aside for GR? (lesson scheduling, frequency)
How are student assessment data used to determine reading groups?
How are GR lessons structured? (i.e. size, duration, lesson components)
What kinds of tasks are other students engaged in while GR groups are running?
Training
What kind of training have you (teachers) received in delivering a Guided Reading program?
Overall impact What do you perceive the impact of GR to be on student achievement in reading?
What is your overall impression of the GR program at L. B. Primary School?
Focus Groups The focus groups will be measuring the short-term outcome: development of a positive attitude towards reading as well as student perceptions of themselves as readers. The questions were developed together with the evaluation ream and the stakeholders.
Focus groups will be made up of a random sample of three groups of 5 students from each classroom who participate regularly in guided reading. The three focus groups will be comprised of students in the same grade: one grade one, one grade two, and one grade three group.
Demographics
Group 1 - All students in this group are in grade one. There are three female students and two male students. The students range in age from 5 years, 7 months, to 6 years 6 months. Within this group, there is one English Language Learner (ELL) and one student with ASD (Autism Spectrum Disorder); the remaining three students are in the regular education program and have no identified special learning needs.
Group 2 - All students in this group are in grade two. There are three female students and two male students. They range in age from 6 years 7 months to 7 years, 6 months. Within this group, there are 2 ELLs and one child with an identified learning disability. The remaining two children are in the regular education program and have no identified special learning needs.
Group 3 - All students in this group are in grade three. There is one female and four males student in this group. They range in age from 7 years 7 months to 8 years, 6 months. Within this group there are no identified special learning needs or ELLs. All students in this group are in the regular education program.
The moderator (evaluator) will record each session for further analysis of the data that the participants are providing. The questions that the moderator will use to lead the discussion are as follows:
Do you read in places other than school? What do you learn about in your guided reading sessions? What helps your reading improve most at school?
The following protocol was developed by the evaluation team and will be used for all three focus groups.
Focus Group Protocol: The moderator greets the students and explains the purpose of the focus group and how it will run.
Introductions of the moderator, the recorder and the students
Our purpose today is to find out about your attitude towards reading as well as your perception of yourself as a reader. I am going to be asking three questions and it is my job to make sure we answer them. There are no right or wrong answers and everything you say is private and confidential.
Here are some rules that we will follow throughout the discussion: 1. Please take turns talking and try not to interrupt each other. 2. Be honest. Your opinion may be different from someone else’s and that is perfectly acceptable. 3. Please try to answer all three questions. 4. Do not use names of teachers or other students in your discussions.
We really appreciate you giving us your honest opinion and participating in this focus group. Before we get started, please help yourself to some juice and cookies.
Quantitative Measures The constructs of the alphabetic principle, phonemic awareness, vocabulary, comprehension, and fluency will be measured. Two different instruments will be used: the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) and the Developmental Reading Assessment (DRA).
The DRA and DIBELS instruments measure students’ basic early literacy skills in different ways and the constructs that appear on the DRA can be linked to one or more of the constructs that appear on the DIBELS:
DRA
DIBELS
Phonological Awareness
Initial Sound Fluency; Phoneme Segmentation
Phonics
Nonsense Word Fluency
Fluency
Oral Reading Fluency
Vocabulary
Word Use Fluency
Comprehension
Retelling Fluency
Teachers will assess each child in grades 1 – 3 on all of these constructs using the DRA, and on all of the prescribed constructs using the DIBELS (e.g. the DIBELS does not recommend testing students in grade three on letter naming, nor students at the beginning of grade one on reading fluency).
Demographics
The classes participating in the evaluation are three straight grade classes: grade one, grade two, and grade three. Each class has a total of 16 students, as descirbed below:
Grade one: 8 females, 8 males; 4 ELLs; ages 5 years, 7 months, to 6 years 6 months; one student has ASD
Grade two: 9 females, 7 males; 2 ELLs; ages 6 years 7 months to 7 years, 6 months; one student with an identified learning disability
Grade three: 11 males, 5 females; no ELLs; 7 years 7 months to 8 years, 6 months; no identified special learning needs
DRA
The Developmental Reading Assessment (DRA) was developed in 1986 by Joetta Beaver and a committee of educators from the Upper Arlington City School District in Ohio (Rathvon, 2006). The intended uses of the DRA include identifying students' reading achievement through the analysis of data to document progress, communicate assessment information, and drive effective reading instruction (Beaver, 2006). The DRA provides a measure of global skill development and readiness for instruction in terms of the reader’s basic literacy skill, including phonemic awareness, phonics, fluency, vocabulary, and comprehension. Student reading of texts in the DRA progresses through the following stages:
introduction to the text: previewing and predicting
oral reading and strategy use
comprehension and response
reading preferences
There is a specific scoring procedure for each of the tasks in the DRA. Procedures for scoring running records of student reading and comprehension tasks are provided in the DRA Observation Guide. The teacher also records observations of reading behaviour as students progress through texts/and or respond to questions. Although scoring guidelines are provided, there remains an element of teacher subjectivity. Therefore, validity and reliability of data are dependent on uniform administration and scoring procedures (Paris & Carpenter, 2003). The researchers will ensure that teachers have undergone training in the administration of the DRA, prior to the commencement of the evaluation, to ensure consistency across administrations. As well, evaluators will engage in observations to ensure fidelity of implementation. The DRA Technical Manual (Pearson, n.d.) reports moderate to high reliability and evidence to support DRA as a valid measure of students' oral reading fluency and comprehension.
The DRA is a criterion-referenced assessment, providing a single source of evidence about students’ basic literacy skills, including phonemic awareness, fluency, vocabulary, and comprehension.
DIBELS
DIBELS (Dynamic Indicators of Basic Early Literacy Skills) was developed by Roland Good and Ruth Kaminski (2007) and their colleagues at the University of Oregon. It is a comprehensive, scientifically based outcomes-driven assessment system to be used for the early identification of students who may need additional support, as well as for the purpose of formatively evaluating and modifying instruction to improve student learning (Kaminski, Cummings, Powell-Smith, & Good, 2008). The reliability and validity of DIBELS have been well documented through ongoing studies and extensive research. The National Center on Student Progress Monitoring (2007) has reported that DIBELS demonstrates sufficient evidence to meet the basic foundational psychometric standards for both reliability and validity.
DIBELS is comprised of a series of short (one minute) fluency measures used to monitor the development of pre-reading and early reading skills. Specifically, DIBELS provides separate measures of students’ basic literacy skills, including phonemic awareness (initial sound fluency, phonemic segmentation); alphabetic principle and phonics (nonsense word fluency, oral reading fluency); vocabulary (word use fluency); and comprehension (retelling fluency, oral reading fluency). Detailed procedures for the administration of each subtest, as well as for maintaining assessment integrity, are provided in the DIBELS Administration and Scoring Guide. After administration, each literacy task is scored and scores on the grouped literacy skill-based task are compiled. The compiled score is considered to be the student’s level of development for the specific basic literacy skill measured.
The researchers will ensure that teachers have undergone training in the administration of the DIBELS, prior to the commencement of the evaluation, to ensure consistency across administrations. As with the DRA assessments, evaluators will engage in observations to ensure fidelity of implementation.
Alison Browne, Robin Halbert, Georgie Perigny, Cavelle Porter, Wanda Quigg
Evaluation Question: Is Guided Reading effective?
Sub questions:
1. Is there fidelity of implementation for the Guided Reading program according to the guidelines established by Fountas and Pinnell (1996)
2. Are students who participate in the program demonstrating improvements in reading skills over the school year?
Qualitative Measures
Semi-Structured Interview
The interview will be used to measure program fidelity. It will address the constructs of organization/ planning, training, and overall impact. Administrators and teachers will be interviewed by the researchers on a one to one basis.
In conducting the semi-structured interviews, we will use a pre-established set of questions to guide the process, yet be open to the order in which questions are asked. An advantage to semi-structured interviews is that they allow interviewers to ask probing questions and to proceed in a manner that has a natural, conversational flow (Leedy & Ormrod, 2010). A disadvantage of the interview process is that researchers are asking participants about events that are in the past, requiring that interviewees rely on their memories, which are often subject to a considerable amount of distortion (Leedy & Ormrod, 2010). Interviews can also take more time than alternate measures, such as the focus group, however as we are only conducting four interviews, we felt that it would provide more insight into each individual’s lived experience than would a focus group situation, in which the limitations of focus groups noted below might come into play.
Demographics of group
Teachers - The three teachers who have Guided Reading programs at the school will be interviewed. These teachers are between the ages of 35 and 45. One teacher has been teaching for 5 years, one for 10 years, and the third for 22 years. All three of the teachers are female and all three have been teaching primary school throughout their careers. The teacher with 22 years experience is currently enrolled in a distance Master's Degree Program, specializing in educational technology. The other two teachers have bachelor's degrees in education, specializing in elementary education. All three teachers have spent their entire careers at L. B. Primary School.
Principal/Administrator - The principal of L. B. Primary School is male, aged 63, and has been with the School Board for 40 years. He recently transferred to L. B. Primary School, having spent all of his career in high schools, first as a high school chemistry teacher, then as a high school principal. He has a Master's degree in Policy and Administration.
There will be 8 questions to measure the constructs.
Interview Questions
Organization and Planning
Training
What kind of training have you (teachers) received in delivering a Guided Reading program?
Overall impact
What do you perceive the impact of GR to be on student achievement in reading?
What is your overall impression of the GR program at L. B. Primary School?
Focus Groups
The focus groups will be measuring the short-term outcome: development of a positive attitude towards reading as well as student perceptions of themselves as readers. The questions were developed together with the evaluation ream and the stakeholders.
Focus groups will be made up of a random sample of three groups of 5 students from each classroom who participate regularly in guided reading. The three focus groups will be comprised of students in the same grade: one grade one, one grade two, and one grade three group.
Demographics
Group 1 - All students in this group are in grade one. There are three female students and two male students. The students range in age from 5 years, 7 months, to 6 years 6 months. Within this group, there is one English Language Learner (ELL) and one student with ASD (Autism Spectrum Disorder); the remaining three students are in the regular education program and have no identified special learning needs.
Group 2 - All students in this group are in grade two. There are three female students and two male students. They range in age from 6 years 7 months to 7 years, 6 months. Within this group, there are 2 ELLs and one child with an identified learning disability. The remaining two children are in the regular education program and have no identified special learning needs.
Group 3 - All students in this group are in grade three. There is one female and four males student in this group. They range in age from 7 years 7 months to 8 years, 6 months. Within this group there are no identified special learning needs or ELLs. All students in this group are in the regular education program.
The moderator (evaluator) will record each session for further analysis of the data that the participants are providing. The questions that the moderator will use to lead the discussion are as follows:
Do you read in places other than school?
What do you learn about in your guided reading sessions?
What helps your reading improve most at school?
The following protocol was developed by the evaluation team and will be used for all three focus groups.
Focus Group Protocol:
The moderator greets the students and explains the purpose of the focus group and how it will run.
Here are some rules that we will follow throughout the discussion:
1. Please take turns talking and try not to interrupt each other.
2. Be honest. Your opinion may be different from someone else’s and that is perfectly acceptable.
3. Please try to answer all three questions.
4. Do not use names of teachers or other students in your discussions.
We really appreciate you giving us your honest opinion and participating in this focus group. Before we get started, please help yourself to some juice and cookies.
Quantitative Measures
The constructs of the alphabetic principle, phonemic awareness, vocabulary, comprehension, and fluency will be measured. Two different instruments will be used: the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) and the Developmental Reading Assessment (DRA).
The DRA and DIBELS instruments measure students’ basic early literacy skills in different ways and the constructs that appear on the DRA can be linked to one or more of the constructs that appear on the DIBELS:
Teachers will assess each child in grades 1 – 3 on all of these constructs using the DRA, and on all of the prescribed constructs using the DIBELS (e.g. the DIBELS does not recommend testing students in grade three on letter naming, nor students at the beginning of grade one on reading fluency).
Demographics
The classes participating in the evaluation are three straight grade classes: grade one, grade two, and grade three. Each class has a total of 16 students, as descirbed below:
Grade one: 8 females, 8 males; 4 ELLs; ages 5 years, 7 months, to 6 years 6 months; one student has ASD
Grade two: 9 females, 7 males; 2 ELLs; ages 6 years 7 months to 7 years, 6 months; one student with an identified learning disability
Grade three: 11 males, 5 females; no ELLs; 7 years 7 months to 8 years, 6 months; no identified special learning needs
DRA
The Developmental Reading Assessment (DRA) was developed in 1986 by Joetta Beaver and a committee of educators from the Upper Arlington City School District in Ohio (Rathvon, 2006). The intended uses of the DRA include identifying students' reading achievement through the analysis of data to document progress, communicate assessment information, and drive effective reading instruction (Beaver, 2006). The DRA provides a measure of global skill development and readiness for instruction in terms of the reader’s basic literacy skill, including phonemic awareness, phonics, fluency, vocabulary, and comprehension. Student reading of texts in the DRA progresses through the following stages:
- introduction to the text: previewing and predicting
- oral reading and strategy use
- comprehension and response
- reading preferences
There is a specific scoring procedure for each of the tasks in the DRA. Procedures for scoring running records of student reading and comprehension tasks are provided in the DRA Observation Guide. The teacher also records observations of reading behaviour as students progress through texts/and or respond to questions. Although scoring guidelines are provided, there remains an element of teacher subjectivity. Therefore, validity and reliability of data are dependent on uniform administration and scoring procedures (Paris & Carpenter, 2003). The researchers will ensure that teachers have undergone training in the administration of the DRA, prior to the commencement of the evaluation, to ensure consistency across administrations. As well, evaluators will engage in observations to ensure fidelity of implementation. The DRA Technical Manual (Pearson, n.d.) reports moderate to high reliability and evidence to support DRA as a valid measure of students' oral reading fluency and comprehension.The DRA is a criterion-referenced assessment, providing a single source of evidence about students’ basic literacy skills, including phonemic awareness, fluency, vocabulary, and comprehension.
DIBELS
DIBELS (Dynamic Indicators of Basic Early Literacy Skills) was developed by Roland Good and Ruth Kaminski (2007) and their colleagues at the University of Oregon. It is a comprehensive, scientifically based outcomes-driven assessment system to be used for the early identification of students who may need additional support, as well as for the purpose of formatively evaluating and modifying instruction to improve student learning (Kaminski, Cummings, Powell-Smith, & Good, 2008). The reliability and validity of DIBELS have been well documented through ongoing studies and extensive research. The National Center on Student Progress Monitoring (2007) has reported that DIBELS demonstrates sufficient evidence to meet the basic foundational psychometric standards for both reliability and validity.
DIBELS is comprised of a series of short (one minute) fluency measures used to monitor the development of pre-reading and early reading skills. Specifically, DIBELS provides separate measures of students’ basic literacy skills, including phonemic awareness (initial sound fluency, phonemic segmentation); alphabetic principle and phonics (nonsense word fluency, oral reading fluency); vocabulary (word use fluency); and comprehension (retelling fluency, oral reading fluency). Detailed procedures for the administration of each subtest, as well as for maintaining assessment integrity, are provided in the DIBELS Administration and Scoring Guide. After administration, each literacy task is scored and scores on the grouped literacy skill-based task are compiled. The compiled score is considered to be the student’s level of development for the specific basic literacy skill measured.
The researchers will ensure that teachers have undergone training in the administration of the DIBELS, prior to the commencement of the evaluation, to ensure consistency across administrations. As with the DRA assessments, evaluators will engage in observations to ensure fidelity of implementation.